Feedback Generation for Performance Problems in Introductory Programming Assignments

Providing feedback on programming assignments manually is a tedious, error prone, and time-consuming task. In this paper, we motivate and address the problem of generating feedback on performance aspects in introductory programming assignments. We studied a large number of functionally correct student solutions to introductory programming assignments and observed: (1) There are different algorithmic strategies, with varying levels of efficiency, for solving a given problem. These different strategies merit different feedback. (2) The same algorithmic strategy can be implemented in countless different ways, which are not relevant for reporting feedback on the student program. We propose a light-weight programming language extension that allows a teacher to define an algorithmic strategy by specifying certain key values that should occur during the execution of an implementation. We describe a dynamic analysis based approach to test whether a student's program matches a teacher's specification. Our experimental results illustrate the effectiveness of both our specification language and our dynamic analysis. On one of our benchmarks consisting of 2316 functionally correct implementations to 3 programming problems, we identified 16 strategies that we were able to describe using our specification language (in 95 minutes after inspecting 66, i.e., around 3%, implementations). Our dynamic analysis correctly matched each implementation with its corresponding specification, thereby automatically producing the intended feedback.Comment: Tech report/extended version of FSE 2014 pape

October Revolution 1917: Causes, course and consequences

Oktobarska revolucija 1917. je jedan od ključnih događaja Ruske povijesti. Sa Oktobarskom revolucijom završava poglavlje Ruskog carstva i kraljevske obitelji Romanov kao vladajuće dinastije, te počinje povijest Sovjetske Rusije i SSSR-a. Prije revolucija iz 1917., značajna je bila ona iz 1905. kojom je car bio prisiljen na ustupke u smislu većih prava za manjine, sazivanja Dume, demokratizacije društva i slabljenja carskih ovlasti. Društvenu koheziju i stanje u državi koje se poboljšalo nakon događaja iz 1905. poremetit će ratna zbivanja, to jest Prvi svjetski rat u kojeg će Rusko Carstvo biti uključeno na strani Antante. Rusko Carstvo će izgubiti veliki broj ljudi u ratu, a to će biti jedan od okidača za novu revoluciju. Nestašica i glad potaknut će građane Petrograda na prosvjede koji će se pretvoriti u Veljačku revoluciju i nakon koje će car abdicirati, a stvoriti će se Privremena vlada pod vodstvom liberalno demokratske partije. Nesposobnost vlade i upornost lijevih struja da država izađe iz rata i provede velike reforme dovest će do novih pobuna. Pod vodstvom Vladimira Lenjina i Lava Trockog započet će Oktobarska revolucija koja će ukinuti Privremenu vladu i stvoriti novo državno uređenje pod vodstvom boljševika. Naposljetku stvorit će se Savez sovjetskih socijalističkih republika, država koja će kroz cijelo 20. stoljeće igrati ulogu svjetske supersile.The October Revolution of 1917 is one of the key events of Russian history. It completes the chapter of the Russian Empire and the Romanov royal family as the ruling dynasty, and thus begins the history of Soviet Russia and the USSR. Prior to the Revolution of 1917, very significant was revolution in 1905, which forced the emperor to make concessions in the sense of greater rights for minorities, summoning Duma, democratizing the society, and weakening emperor powers. The social cohesion and the state of the country that improved after the 1905 events, will be disrupted by the war, that is, the First World War, in which the Russian Empire will be included on the side of Triple Entente. The Russian Empire will lose a large number of people in the war, which will be one of the triggering factors for the new revolution. Poverty and hunger will encourage citizens of Petrograd to protest, and that, will turn into the February Revolution, after which the emperor will abdicate, and the Provisional Government will be formed under the leadership of the Liberal Democratic Party. The government's inability and persistence of left-wing currents to stop the country from participating in the war and carry out major reforms, will lead to new rebellions. Under the leadership of Vladimir Lenin and Leon Trotsky, the October Revolution will begin, and it will abolish the Provisional Government and create a new state government under the leadership of the Bolsheviks. Ultimately, the Union of Soviet Socialist Republics will be formed, a country that will play the role of the world superpower throughout the 20th century

October Revolution 1917: Causes, course and consequences

Oktobarska revolucija 1917. je jedan od ključnih događaja Ruske povijesti. Sa Oktobarskom revolucijom završava poglavlje Ruskog carstva i kraljevske obitelji Romanov kao vladajuće dinastije, te počinje povijest Sovjetske Rusije i SSSR-a. Prije revolucija iz 1917., značajna je bila ona iz 1905. kojom je car bio prisiljen na ustupke u smislu većih prava za manjine, sazivanja Dume, demokratizacije društva i slabljenja carskih ovlasti. Društvenu koheziju i stanje u državi koje se poboljšalo nakon događaja iz 1905. poremetit će ratna zbivanja, to jest Prvi svjetski rat u kojeg će Rusko Carstvo biti uključeno na strani Antante. Rusko Carstvo će izgubiti veliki broj ljudi u ratu, a to će biti jedan od okidača za novu revoluciju. Nestašica i glad potaknut će građane Petrograda na prosvjede koji će se pretvoriti u Veljačku revoluciju i nakon koje će car abdicirati, a stvoriti će se Privremena vlada pod vodstvom liberalno demokratske partije. Nesposobnost vlade i upornost lijevih struja da država izađe iz rata i provede velike reforme dovest će do novih pobuna. Pod vodstvom Vladimira Lenjina i Lava Trockog započet će Oktobarska revolucija koja će ukinuti Privremenu vladu i stvoriti novo državno uređenje pod vodstvom boljševika. Naposljetku stvorit će se Savez sovjetskih socijalističkih republika, država koja će kroz cijelo 20. stoljeće igrati ulogu svjetske supersile.The October Revolution of 1917 is one of the key events of Russian history. It completes the chapter of the Russian Empire and the Romanov royal family as the ruling dynasty, and thus begins the history of Soviet Russia and the USSR. Prior to the Revolution of 1917, very significant was revolution in 1905, which forced the emperor to make concessions in the sense of greater rights for minorities, summoning Duma, democratizing the society, and weakening emperor powers. The social cohesion and the state of the country that improved after the 1905 events, will be disrupted by the war, that is, the First World War, in which the Russian Empire will be included on the side of Triple Entente. The Russian Empire will lose a large number of people in the war, which will be one of the triggering factors for the new revolution. Poverty and hunger will encourage citizens of Petrograd to protest, and that, will turn into the February Revolution, after which the emperor will abdicate, and the Provisional Government will be formed under the leadership of the Liberal Democratic Party. The government's inability and persistence of left-wing currents to stop the country from participating in the war and carry out major reforms, will lead to new rebellions. Under the leadership of Vladimir Lenin and Leon Trotsky, the October Revolution will begin, and it will abolish the Provisional Government and create a new state government under the leadership of the Bolsheviks. Ultimately, the Union of Soviet Socialist Republics will be formed, a country that will play the role of the world superpower throughout the 20th century

Automated feedback generation in introductory programming education : a dynamic program analysis approach

Anyone who ever manually corrected or graded any type of student assignment is aware of how tedious, error-prone and time-consuming this task is. In the context of programming assignments the situation is even worse: with a rising demand for programming education there are online courses with thousands of students, which makes manual grading downright impossible. However, students who are learning programming, especially at the introductory level, are in pressing need of guidance. This has motivated a lot of research on computer-aided feedback generation in introductory programming education. Many of these approaches are based on practices and methods employed in software engineering, testing, and verification, such as: code reviews, debugging, automated test generation, program synthesis, program repair, ... This is very useful feedback as it mimics how software engineers reason about the code. However, there are still numerous open issues. In this thesis, we will address the following two challenges: Most of the existing research focused on generating functional feedback, that is, to help the students write any correct program, while ignoring non-functional properties such as e.g., performance. Most of the existing approaches lack some of the following desirable properties: automation (require as little manual effort as possible), correctness (generate correct feedback), performance (generate feed-back in order of seconds), exhaustiveness (generate feedback for most student programs), and usefulness (reduce the teacher effort or help the student to make progress). We first study inefficient (but correct) student programs in order to understand the performance problems faced by students in introductory programming. From the study we observe that in order to provide feedback we need to identify the high-level algorithmic strategy used by the student's program, while ignoring its low-level implementation details. To that end we propose a lightweight specification language that enables the teacher to specify various algorithmic strategies, and a novel dynamic program analysis to determine whether a student's attempt matches the teacher's specification. To provide feedback on (functional) correctness of student programs we propose a novel fully-automated program repair algorithm that uses the wisdom of the crowd: it leverages the existing correct student solutions to syntactically modify incorrect student attempts. The repair algorithm builds on and extends our earlier dynamic program analysis in order to determine whether a modified (repaired) program has the same behavior as a correct solution. We have implemented both of the proposed approaches in practical tools and performed an experimental evaluation on a large number of student attempts from various programming courses. We have evaluated both of our approaches on the above mentioned desirable properties: automation, correctness, performance, exhaustiveness, and usefulness.14

IMMIGRANTS AND THE SHAPING OF THE AMERICAN SOCIETY: NEW YORK IN 19TH CENTURY

Razdoblje 19. stoljeća za Sjedinjene Države značilo je ekonomsku, političku i kulturnu ekspanziju. Imigracijski procesi koji su prisutni kroz cijelo stoljeće obilježili su Sjedinjene Države stvorivši od njih veliki melting pot poglavito europskih imigranata. New York je tada bio prva stanica koju je velika većina imigranata prolazila na putu k ostalim dijelovima Sjedinjenih Država, ali također New York je bio i grad koji su imigranti oblikovali i stvorili od njega metropolis i najveći grad Sjedinjenih Država. Sjedinjene su Države kroz razne etape 19. stoljeća imale razne regulacije po pitanju imigracije, ali uvijek su kao država bile u potrebi za velikim brojem radne snage koja je stizala iz Europe. Imigranti su odlazili iz Europe najviše zbog ekonomskih razloga, ali i raznih progona, ratova i prirodnih nepogoda. Imigranti su u New Yorku bili suočeni s novinama s kojima se nisu susretali u svojim matičnim državama jer su od samog mukotrpnog puta pa onda sve do života i rada u New Yorku morali naučiti živjeti s ljudima raznog porijekla i statusa. Valovi imigracije podijeljeni na dva velika vala predstavljaju i jasan pokazatelj kako se i kojom brzinom koji dio Europe razvijao ili bio suočen s ekonomskim i socijalnim problemima. Četiri velike skupine imigranata obilježile su te etape. Prvi val s Nijemcima i Ircima, koji su uz prve imigrante iz kolonijalnog doba stvorili političke i društvene temelje grada, i drugi val sa Židovima i Talijanima, koji su ga „izgradili“ i kulturološki obilježili do danas.The 19th century period meant economic, political, and cultural expansion for the United States. The immigration processes that have been present throughout the century have marked the United States, making it a great melting pot, especially because of European immigrants. New York was the first step into a new nation, a city where the vast majority of immigrants passed on their way to the other parts of the United States. But New York was also the city that immigrants shaped and created, forming a metropolis and largest city of the United States. Throughout the various stages of the 19th century, the United States had different regulations regarding immigration. But as a country, the United States were always in need of a large number of labor force, coming from Europe and other parts of the world. Immigrants left Europe mostly for economic reasons, but also because of various persecutions, wars and natural disasters. Immigrants in New York were faced with new challenges that they had not encountered in their home countries, from the arduous journey to the life and work in New York where they had to learn to live with people of various backgrounds and statuses. The waves of immigration are divided into two major ones. These waves are also a clear indicator of how and at what rate which part of Europe was developing or was facing economic and social problems. Four large groups of immigrants marked these stages. The first wave, with German and Irish immigrants who, along with the first immigrants from the colonial era, created the political and social foundations of the city, and the second wave with the Jews and Italians who had "built" and culturally marked it to this day

IMMIGRANTS AND THE SHAPING OF THE AMERICAN SOCIETY: NEW YORK IN 19TH CENTURY

Razdoblje 19. stoljeća za Sjedinjene Države značilo je ekonomsku, političku i kulturnu ekspanziju. Imigracijski procesi koji su prisutni kroz cijelo stoljeće obilježili su Sjedinjene Države stvorivši od njih veliki melting pot poglavito europskih imigranata. New York je tada bio prva stanica koju je velika većina imigranata prolazila na putu k ostalim dijelovima Sjedinjenih Država, ali također New York je bio i grad koji su imigranti oblikovali i stvorili od njega metropolis i najveći grad Sjedinjenih Država. Sjedinjene su Države kroz razne etape 19. stoljeća imale razne regulacije po pitanju imigracije, ali uvijek su kao država bile u potrebi za velikim brojem radne snage koja je stizala iz Europe. Imigranti su odlazili iz Europe najviše zbog ekonomskih razloga, ali i raznih progona, ratova i prirodnih nepogoda. Imigranti su u New Yorku bili suočeni s novinama s kojima se nisu susretali u svojim matičnim državama jer su od samog mukotrpnog puta pa onda sve do života i rada u New Yorku morali naučiti živjeti s ljudima raznog porijekla i statusa. Valovi imigracije podijeljeni na dva velika vala predstavljaju i jasan pokazatelj kako se i kojom brzinom koji dio Europe razvijao ili bio suočen s ekonomskim i socijalnim problemima. Četiri velike skupine imigranata obilježile su te etape. Prvi val s Nijemcima i Ircima, koji su uz prve imigrante iz kolonijalnog doba stvorili političke i društvene temelje grada, i drugi val sa Židovima i Talijanima, koji su ga „izgradili“ i kulturološki obilježili do danas.The 19th century period meant economic, political, and cultural expansion for the United States. The immigration processes that have been present throughout the century have marked the United States, making it a great melting pot, especially because of European immigrants. New York was the first step into a new nation, a city where the vast majority of immigrants passed on their way to the other parts of the United States. But New York was also the city that immigrants shaped and created, forming a metropolis and largest city of the United States. Throughout the various stages of the 19th century, the United States had different regulations regarding immigration. But as a country, the United States were always in need of a large number of labor force, coming from Europe and other parts of the world. Immigrants left Europe mostly for economic reasons, but also because of various persecutions, wars and natural disasters. Immigrants in New York were faced with new challenges that they had not encountered in their home countries, from the arduous journey to the life and work in New York where they had to learn to live with people of various backgrounds and statuses. The waves of immigration are divided into two major ones. These waves are also a clear indicator of how and at what rate which part of Europe was developing or was facing economic and social problems. Four large groups of immigrants marked these stages. The first wave, with German and Irish immigrants who, along with the first immigrants from the colonial era, created the political and social foundations of the city, and the second wave with the Jews and Italians who had "built" and culturally marked it to this day